🕷️️ Job Radar • SCRAPING

freelancer.com 2026-04-18 🟡

🔹 Automated Data Fetching for Amazon & Flipkart ASINs
👤 Client: 🇮🇳 Mumbai, India Member since 2026-04-04
💰 Price: $76 Average bid
🚩 Problem: Need reliable, non-blocking crawlers to fetch product data from Amazon and Flipkart.
📦 Existing: Not specified.

Specifications:

[Target] Crawl Amazon and Flipkart for specific ASINs.
[Method] Use headless browser automation with Python (Scrapy or Playwright) for non-blocking operations.
[UI/UX] No UI required; focus on backend data collection and self-healing logic.
[Stack] Python, Scrapy/Playwright, MySQL, SMTP library for email alerts.
[Security] Implement rate limiting to avoid detection, use secure credentials storage.
[Format] Store data in a structured MySQL database.

Workflow:

1. Set up separate projects for Amazon and Flipkart crawlers with non-blocking architecture.
2. Define ASINs to be crawled using environment variables or configuration files.
3. Implement self-healing logic to handle interruptions like captchas, rate limits, network drops.
4. Schedule the crawlers to run every 24 hours using cron jobs or systemd timers.
5. Write clean, well-commented code for both crawlers and include error handling.
6. Create a MySQL schema and migration scripts for data storage.
7. Develop an email notification module with SMTP settings.
8. Implement logging to track the health of each crawler and any errors encountered.